feat(pipeline): allow syncing blocks ontop of the proposed chain by Maddiaa0 · Pull Request #21025 · AztecProtocol/aztec-packages

Maddiaa0 · 2026-03-03T02:17:30Z

Overview

Key contributions:

In the pr above feat(pipeline): introduce pipeline views for building #21026 publishing was a blocking action, in this pr we move publishing to be a non blocking option, there a publisher can schedule when it should start trying to publish a block.
This keeps track of valid checkpoints that are pending and not settled to L1 - and allows building ontop of them.

Adds a second p2p callback that separates what runs for all nodes / validator nodes

Testing

epochs_mbps.pipeline now expects 3 blocks per checkpoint, just like the original epochs_mbps test, now it is fully pipelined.

Upcoming

updating the timetable to allow for longer time building in the slot - this pr does not extend the time allocated to block building.
handing rollbacks when the pendingCheckpoint needs to be rolled back / cleared.

Maddiaa0 · 2026-03-03T20:51:20Z

yarn-project/archiver/src/modules/validation.ts

yarn-project/archiver/src/modules/data_store_updater.ts

yarn-project/sequencer-client/src/sequencer/sequencer.ts

yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts

yarn-project/validator-client/src/block_proposal_handler.ts

yarn-project/archiver/src/modules/data_source_base.ts

yarn-project/archiver/src/modules/l1_synchronizer.ts

yarn-project/archiver/src/store/block_store.ts

yarn-project/aztec-node/src/aztec-node/server.ts

yarn-project/sequencer-client/src/global_variable_builder/global_builder.ts

yarn-project/sequencer-client/src/sequencer/sequencer.ts

fix

yarn-project/validator-client/src/proposal_handler.ts

yarn-project/ethereum/src/contracts/rollup.test.ts

BEGIN_COMMIT_OVERRIDE chore: setup snapshot retention rules (#22107) fix: add branding to Buffer16 and Buffer32 to prevent cross-type misuse (#22128) chore: (A-723) redact beacon api key in debug path (#22115) fix(archiver): use Buffer16.ZERO for rolling hash fallback in L1 synchronizer (#22133) feat(pipeline): allow syncing blocks ontop of the proposed chain (#21025) fix(logger): no arbitrary long logger module name (#22143) chore: setup workload identity for ESO (#22137) fix: Make test more robust to committee selection. (#22139) fix: handle error event on SSH agent socket to prevent Node crash (#22090) fix: disable SponsoredFPC on staging-public (#22150) fix(archiver): swallow error when rollup contract not yet finalized on L1 (#22156) chore: update dashboard (#22157) chore: switch testnet back to prod resource profile (#22159) chore: nightly scenario deployments should not be triggered by older images (#22148) END_COMMIT_OVERRIDE

Maddiaa0 force-pushed the md/pipelining-syncing branch from 8de6157 to 31f941d Compare March 3, 2026 20:51

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from 3a75fdb to 1ca98d8 Compare March 3, 2026 20:51

This was referenced Mar 3, 2026

feat(sequencer): add build-ahead config and metrics #20779

Merged

feat(pipeline): introduce pipeline views for building #21026

Merged

chore(pipeline): timetable adjustments under pipelining #21076

Open

spalladino reviewed Mar 4, 2026

View reviewed changes

yarn-project/archiver/src/modules/validation.ts Show resolved Hide resolved

yarn-project/archiver/src/modules/data_store_updater.ts Outdated Show resolved Hide resolved

spalladino reviewed Mar 4, 2026

View reviewed changes

yarn-project/sequencer-client/src/sequencer/sequencer.ts Outdated Show resolved Hide resolved

spalladino reviewed Mar 4, 2026

View reviewed changes

yarn-project/sequencer-client/src/sequencer/checkpoint_proposal_job.ts Show resolved Hide resolved

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 6a16e98 to 28a0520 Compare March 6, 2026 17:19

Maddiaa0 commented Mar 9, 2026

View reviewed changes

yarn-project/validator-client/src/block_proposal_handler.ts Outdated Show resolved Hide resolved

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 27f8a4b to f5c6308 Compare March 9, 2026 12:12

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from b1b1b14 to ce0e422 Compare March 9, 2026 12:12

Maddiaa0 force-pushed the md/pipelining-syncing branch from f5c6308 to ae6b651 Compare March 9, 2026 12:26

Maddiaa0 marked this pull request as ready for review March 9, 2026 12:29

Maddiaa0 mentioned this pull request Mar 9, 2026

feat(pipeline): handle pipeline prunes #21250

Open

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from ce0e422 to d8b8d31 Compare March 9, 2026 16:01

Maddiaa0 force-pushed the md/pipelining-syncing branch 2 times, most recently from 3a47fbf to 4bb6845 Compare March 9, 2026 16:07

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from d8b8d31 to 6973d24 Compare March 9, 2026 16:41

Maddiaa0 force-pushed the md/pipelining-syncing branch from 4bb6845 to 0e9b52e Compare March 9, 2026 16:41

Maddiaa0 mentioned this pull request Mar 10, 2026

chore(pipeline): spartan config #21285

Open

Maddiaa0 force-pushed the md/pipelining-syncing branch from 0e9b52e to dee426a Compare March 12, 2026 15:52

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from 6973d24 to 21cdcd5 Compare March 12, 2026 15:52

This was referenced Mar 12, 2026

feat(pipeline): minimize deadzone w cross slot attesting #21435

Open

chore: Update latency explorer #21590

Open

chore(pipeline): shorten last block in checkpoint build duration to allow checkpoint construction #21591

Open

Maddiaa0 force-pushed the md/update-epoch-cache-for-buildahead branch from 21cdcd5 to 9543a61 Compare March 16, 2026 18:08

Maddiaa0 force-pushed the md/pipelining-syncing branch from 7802979 to b371deb Compare March 25, 2026 15:59

Maddiaa0 changed the base branch from graphite-base/21025 to merge-train/spartan March 25, 2026 15:59

spalladino reviewed Mar 26, 2026

View reviewed changes

Maddiaa0 added 14 commits March 27, 2026 16:14

feat(pipeline): allow syncing blocks ontop of the proposed chain

4b98586

chore: dont build too far ahead

b22d4ae

chore: checkpoint propsal validator

e3cea29

chore: more checks in block store

0a85c32

chore: flip invalidation condition + bit more

00b761c

chore: post merge fix

270cc95

fix: cleanup ha test

1f92a49

chore: pending checkpoint should not return undefined

54d2b44

chore: update pending checkpoint -> proposed checkpoint

69d03e6

chore: merge checkpoint proposal handler into proposal handler

735003c

chore: include archive and outHash in proposedCheckpointData

30605d2

fi

e4738d7

fix

a6c4c47

pr review fixes

69756aa

fix

Maddiaa0 force-pushed the md/pipelining-syncing branch from bfccd52 to 69756aa Compare March 27, 2026 18:00

Maddiaa0 added 2 commits March 27, 2026 18:35

fix

5a00480

fix archiver interface test

c8b2d37

Maddiaa0 commented Mar 27, 2026

View reviewed changes

yarn-project/validator-client/src/proposal_handler.ts Show resolved Hide resolved

Maddiaa0 added 3 commits March 27, 2026 19:15

fix review

6b8d140

fix review 2

9099a5a

fix: validators always re-execute

6eed407

spalladino approved these changes Mar 27, 2026

View reviewed changes

yarn-project/ethereum/src/contracts/rollup.test.ts Show resolved Hide resolved

Maddiaa0 enabled auto-merge (squash) March 30, 2026 14:13

Maddiaa0 merged commit 2077d27 into merge-train/spartan Mar 30, 2026
12 checks passed

Maddiaa0 deleted the md/pipelining-syncing branch March 30, 2026 14:14

AztecBot mentioned this pull request Mar 30, 2026

feat: merge-train/spartan #22132

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

feat(pipeline): allow syncing blocks ontop of the proposed chain#21025

feat(pipeline): allow syncing blocks ontop of the proposed chain#21025
Maddiaa0 merged 19 commits intomerge-train/spartanfrom
md/pipelining-syncing

Maddiaa0 commented Mar 3, 2026 •

edited

Loading

Uh oh!

Maddiaa0 commented Mar 3, 2026 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Maddiaa0 commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Overview

Testing

Upcoming

Uh oh!

Maddiaa0 commented Mar 3, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Maddiaa0 commented Mar 3, 2026 •

edited

Loading

Maddiaa0 commented Mar 3, 2026 •

edited

Loading